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PLAN  FOR  FIELD  VALIDATION  TRIALS 
OF  CRITICAL  COMBAT  FUNCTION  ASSESSMENT  PACKAGES 

1.  INTRODUCTION 


A.  Background 

In  1992  the  Army  Research  Institute  initiated  the  development  of  an  Integrated  Program 
for  Combined  Arms  Training  based  on  an  analysis  of  the  functions  performed  by  combat  units. 
One  motivation  for  basing  training  on  a  functional  analysis  was  to  move  away  from  the 
traditional  task-based  model  for  collective  training,  toward  a  model  based  on  functions  performed 
on  the  battlefield.  The  idea  was  that  such  a  training  program  would  be  robust  against  changes 
in  the  underlying  doctrine,  the  supporting  tactics,  techniques  and  procedures,  as  well  as  changes 
to  equipment.  The  Army  had  based  some  training  guidance  on  the  Battlefield  Operating  Systems 
(BOS),  but  these  had  been  found  to  be  too  large  in  scale  for  effective  management.  On  the  other 
hand,  the  tasks  in  Army  Mission  Training  Plans  (MTPs)  seemed  to  be  too  fine  in  scale.  The 
Critical  Combat  Functions  (CCFs)  were  first  identified  as  BOS  sub-functions,  focusing  on 
specific  units  or  echelons  that  had  to  perform  particular  functions  within  a  BOS.  Eventually 
CCFs  were  defined  as; 

The  integration  of  related  participants  and  tasks  that  represent  a  source  of  combat 

power.  The  synchronization  of  critical  combat  functions  provides  commanders  at 

any  echelon  with  a  definable  outcome  that  materially  affects  the  battle*. 

Of  39  Critical  Combat  Functions,  25  are  directly  related  to  the  performance  of  missions 
by  maneuver  task  forces  and  were  selected  for  further  development.  A  top-down  front-end- 
analysis  was  conducted  which  resulted  in  several  products,  including: 

•  a  statement  of  the  purpose  for  each  CCF, 

•  a  list  of  outcomes  that  must  be  accomplished  to  achieve  the  purpose,  and 

•  a  consolidated  list  of  tasks  (drawn  from  many  Army  sources)  required  to  achieve  the 
outcomes. 

These  task  lists  were  organized  around  the  chronology  of  battle  phases  and  the  groupings  of 
personnel  who  performed  them.  These  task  lists  have  been  subjected  to  considerable  internal  (to 
the  ARI  contract  team)  and  external  review. 

The  Integrated  Program  for  Combined  Arms  Training  was  to  include  guidance  to  assist 
commanders  in  the  design  of  training  events  to  allow  personnel  to  practice  their  tasks.  As  this 


'  This  definition  is  found  in  the  Preface  to  each  CCF  Task  Analysis.  See,  for  example, 
Mcllroy  (1994). 
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development  was  being  conceptualized,  it  became  clear  that  a  set  of  assessment  packages  would 
be  useful  to  aid  the  commander  in  determining  the  status  of  his  unit.  Subsequently,  ARI 
contracted  with  the  team  developing  the  CCFs  to  develop  CCF-based  assessment  packages. 

Although  the  assessment  packages  were  developed  to  assist  home  station  commanders 
with  the  management  of  training,  another  audience  was  also  recognized:  Observer/controllers  at 
Combat  Training  Centers  (CTCs)  could  use  the  assessment  packages  to  make  standardized 
appraisals  of  the  performance  of  units.  These  two  uses  are  discussed  in  more  detail  in  the 
following. 

Home  station  commanders  use  the  Army’s  Training  Management  Cycle  (FM  25-100), 
depicted  in  Figure  1,  to  manage  unit  training.  The  assessment  packages  developed  in  this  study 
may  be  used  in  three  ways  as  part  of  the  Training  Management  Cycle: 

1)  They  may  serve  as  an  aid  to  the  commander  who 
is  trying  to  judge  the  status  of  his  unit  at  the  start  of  the 
training  management  cycle.  A  commander  who  had  used  the 
assessment  packages  in  prior  exercises  would  have  the 
information  needed  to  perform  the  assessment  in  accordance 
with  FM  25-100.  Using  these  packages  to  systematically 
record  assessments  would  provide  valuable  information  for 
a  new  commander  when  he  assumes  command. 

2)  They  may  serve  as  a  guide  for  the  information  to 
be  gathered  during  home  station  training  exercises  to 
determine  whether  the  training  has  been  effective  in  bringing 
a  unit  up  to  the  level  of  proficiency  the  commander  desires. 

The  emphasis  here  is  on  diagnosing  weaknesses  in 
performance  so  that  efficient  and  effective  retraining 
activities  may  be  developed.  The  assessment  packages  allow 
the  diagnosis  to  identify  the  personnel  and  tasks  most  in 
need  of  retraining;  thus,  retraining  activities  can  be  focused  to  conserve  resources  and  time. 

3)  By  comparing  unit  performance  to  a  prior  baseline,  the  assessment  packages  could  also 
be  used  to  document  particularly  successful  tactics,  techniques  and  procedures  that  the  unit  may 
have  developed,  or  to  identify  effective  training  activities. 

The  primary  mission  of  the  CTCs  is  to  provide  feedback  to  units  about  their  performance 
under  highly  realistic  and  demanding  conditions.  The  secondary  mission  is  to  provide 
information  to  the  Army  about  the  performance  of  units.  Performing  these  missions  effectively 
requires  that  the  assessments  of  unit  performance  be  standardized  with  respect  to  content  and  as 
free  of  subjective  judgement  as  possible.  The  CCF-based  assessment  packages  would  help  to 
assure  that  CTC  assessments  have  these  qualities  in  three  ways: 
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Figure  1.  Training 
Management  Cycle 
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1)  They  would  help  to  standardize  the  assessments  made  at  all  CTCs  by  providing  a 
common  framework  for  observing  performance  and  recording  the  assessments. 

2)  They  could  help  to  control  for  variations  among  OCs  in  assessing  equivalent 
performances  through  their  focus  on  observable  facets  of  performance,  thus  reducing  the 
subjective  component. 

3)  They  could  be  used  as  a  basis  for  training  the  OCs  to  help  standardize  their 
assessments  in  terms  of  content  and  reliability  (defined  as  between-OC  agreement). 

With  respect  to  the  primary  mission  of  the  CTCs,  using  the  CCF-based  assessment 
packages  to  standardize  the  information  provided  to  the  training  units  would  allow  those  units 
to  compare  their  self-assessments  (typically  made  under  less  demanding  conditions)  to  the  CTC’s 
assessments  as  a  check  on  their  understanding  of  what  is  required  to  accomplish  a  particular 
mission.  The  training  units  could  then  develop  more  effective  training  programs.  The 
assessment  packages  could  become  a  vehicle  for  the  CTC  OCs  to  use  in  mentoring  their 
counterparts  at  home  station,  as  suggested  by  the  former  Chief  of  Operations  Group  at  NTC,  BG 
J.  O’Neal  (1994). 

The  second  mission  of  the  CTCs  is  to  provide  information  to  the  Army  about  the 
performance  of  units.  The  assessment  packages  could  be  used  to  identify  particularly  effective 
tactics,  techniques  and  procedures  employed  by  the  training  units.  These  could  be  distributed  to 
the  Army  through  the  Lessons  Learned  publications.  Using  standardized  assessments,  gathered 
over  time,  the  Army  could  evaluate  the  effects  of  changes  to  Army  doctrine,  organization, 
training,  materiel,  leadership,  or  soldier  quality.  Some  analysis  strategies  that  could  be  adapted 
to  this  purpose  are  described  by  Keesling  and  Clifton  (1994). 


B.  Purpose 

The  purpose  of  this  document  is  to  present  a  plan  to  provide  field  validation  trials  of  the 
CCF  Assessment  Packages.  A  preliminary  review  that  should  focus  on  the  degree  to  which  the 
assessment  packages  reflect  current  Army  doctrine  is  also  suggested.  The  field  trials  should 
focus  on  practical  problems  in  implementing  the  assessment  packages  at  CTCs,  major  simulation 
centers,  and  home  stations.  The  first  section  of  this  document  briefly  describes  the  structure  and 
content  of  the  CCF  Assessment  Packages.  The  next  section  describes  the  plan  for  review  and 
field  trials. 


II.  CONTENT  OF  THE  ASSESSMENT  PACKAGES 
The  Assessment  Packages  have  four  major  sections: 

1.  The  statement  of  purpose.  This  is  the  purpose  of  the  CCF,  the  end  state  that 
particular  function  is  to  achieve.  The  primary  question  that  must  be  addressed  after 
each  mission  is,  “Did  the  unit  achieve  the  purpose?”  This  section  asks  for  a  simple 


3 


yes  or  no  response  to  this  question.  In  addition,  if  the  unit’s  performance  was 
particularly  excellent,  this  section  provides  a  place  for  the  observer  to  record  the 
tactics,  techniques  and  procedures  the  unit  used  for  possible  dissemination  to  other 
Army  units. 

2.  Measures  of  effectiveness.  This  section  of  the  assessment  package  contains  tables  and 
checklists  that  may  be  used  to  record  objective  --  sometimes  quantitative  ~  measures 
of  performance.  These  measures  are  oriented  to  events  on  the  battlefield.  Examples 
are  the  amount  of  time  it  takes  to  prepare  an  order,  the  number  of  fighting  positions 
that  are  prepared  to  Axmy  standard,  or  whether  information  was  disseminated  in  a 
timely  manner. 

3.  Assessment  strategy.  In  most  training  exercises,  particularly  at  home  stations,  there 
are  barely  sufficient  numbers  of  observers  who  can  monitor  the  activities.  This 
section  of  the  assessment  package  indicates  when  and  where  the  observer  is  most 
likely  to  find  the  information  needed  to  make  a  particular  assessment.  This  section 
is  a  job  aid  for  the  personnel  managing  the  exercise,  allowing  them  to  allocate 
observers  to  monitor  the  key  locations,  events,  products  and  personnel. 

4.  Assessment  and  diagnosis.  In  this  section  of  the  assessment  package  each  outcome 
is  addressed  in  two  subsections.  The  first  asks  for  an  overall  assessment  of  the 
performance  of  the  outcome.  ‘Assessment  statements’  are  provided  to  orient  the 
observer  to  observable  performances  related  to  the  tasks  underlying  the  outcome.  The 
second  subsection  asks  for  a  more  detailed  diagnosis  of  any  problems  in  performance 
with  respect  to  this  outcome.  In  the  diagnosis  subsection,  the  most  important  tasks 
are  given  in  more  detail  (called  ‘task  elements’)  so  that  the  observer  may  note  those 
that  are  particularly  deficient.  (Or,  highlight  those  that  are  exemplary  to  document 
a  particularly  effective  tactic,  technique  or  procedure.) 

Table  1  shows  the  page  describing  the  generic  assessment  scale  that  is  incorporated  in 
each  of  the  assessment  packages.  Attempts  to  define  specific  behavioral  anchors  for  each 
outcome  assessment  were  not  successful.  The  outcomes  result  from  the  integrated  performance 
of  many  tasks.  The  tasks  may  vary  in  importance  from  mission  to  mission  depending  on  the 
nature  of  the  threat  or  the  fortunes  of  war.  It  was  not  possible  to  describe  each  anchor  point 
adequately  without  resort  to  artificial  and  arbitrary  demarcations  based  on,  for  example,  the 
percentage  of  tasks  performed  to  standard.  Statements  of  this  nature  might  easily  be 
misinterpreted  as  Army  standards  of  performance,  and  this  project  was  not  intended  to  develop 
such  standards.  An  important  subsequent  development  of  these  packages  will  be  the  tailoring  of 
the  behavioral  anchors  to  each  outcome.  An  Army  agency  should  define  these  behavioral 
anchors.  For  example,  where  Army  standards  have  been  established  they  could  be  used  to  define 
the  ‘Adequate’  level  of  performance.  The  appropriate  Army  school(s)  could  promulgate  these 
behavioral  anchors.  OCs  from  the  CTCs  might  be  able  to  provide  suggestions  (or  review  of 
proposed  anchors)  through  the  Center  for  Army  Lessons  Learned. 
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Table  1:  Assessment  Scale 
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behavioral  anchors.  OCs  from  the  CTCs  might  be  able  to  provide  suggestions  (or  review  of  proposed  anchors)  through  the  Center 
for  Army  Lessons  Learned.] 


Generally  speaking,  the  assessment  packages  synopsize  the  tasks  presented  in  the  task 
analysis.  The  task  analysis  and  training  guidance  package  for  each  CCF  present  the  tutorial 
information  about  how  the  CCF  is  laid  out  and  how  it  connects  to  the  other  CCFs.  The 
assessment  packages  focus  on  the  principal  tasks  associated  with  each  outcome  of  each  CCF. 


III.  PLAN  FOR  REVIEW  AND  FIELD  TRIAL 


A.  Overview 

The  following  plan  describes  four  coordinated  approaches  to  evaluating  the  CCF 
Assessment  Packages.  The  first  approach  addresses  the  need  to  be  sure  that  the  CCF  Assessment 
Packages  conform  to  current  Army  doctrine.  The  remaining  approaches  address  the  practicality 
of  implementing  the  packages  in  various  settings.  Three  settings  are  considered:  CTCs,  Home 
Stations  and  major  simulation  centers.  The  integration  of  these  results  is  addressed  in  the  last 
section. 

B.  Doctrinal  Review 


Although  the  CCF  Task  Analyses  have  xmdergone  extensive  peer  review,  they  have  not 
been  systematically  reviewed  at  the  Army  school(s)  most  responsible  for  the  associated  doctrine. 
A  review  of  the  Task  Analysis  and  the  Assessment  Package  associated  with  each  CCF  by  a 
doctrinal  specialist  should  validate  the  conformance  of  the  Task  Analysis  with  current  doctrine 
and  assure  that  the  Assessment  Package  reflects  the  most  salient  tasks  to  be  addressed  in  training. 
It  would  be  particularly  useful  to  have  the  Army  schools  develop  behavioral  anchors  for  ratings 
of  each  of  the  outcome  assessments. 

This  review  could  occur  prior  to  the  field  trials,  or  concurrently.  The  benefit  of  having 
the  review  occur  first  is  that  the  materials  fielded  to  the  units  and  the  CTCs  would  be  revised  to 
reflect  the  current  doctrine.  If  the  review  is  conducted  concurrently  with  the  field  trials,  then 
questions  about  doctrinal  applicability  raised  in  the  field  could  not  be  answered  until  the  review 
was  completed. 

C.  Home  Station  Field  Trials 


The  purpose  of  the  home  station  field  trials  is  to  determine  the  practicality  of  the 
assessment  packages.  Typically  home  stations  are  short  on  observer/controller  resources, 
especially  in  support  of  task  force  sized  exercises.  The  commander  will  have  to  allocate  his 
observer  resources  to  gather  the  most  useful  information  about  unit  performance.  Home  stations 
also  implement  a  training  cycle  which  starts  with  training  smaller  elements  on  fundamental  skills 
and  progresses  to  train  larger  aggregates  of  smaller  units  on  increasingly  complex  tasks.  An 
entire  task  force  or  brigade  participates  in  a  limited  number  of  exercises.  The  commander  will 
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have  to  identity  the  sections  of  the  assessment  packages  that  axe  most  relevant  to  the  training  at 
each  stage  of  this  process.  The  questions  to  be  answered  by  these  field  trials  include: 

•  Can  the  commander  use  the  assessment  packages  as  the  basis  for  gathering  the 
information  he  needs  throughout  the  FM  25-100  training  management  cycle? 

•  Can  the  commander  easily  determine  which  components  of  the  assessment  packages  are 
useful  for  assessing  training  of  staff  elements  and  subordinate  units? 

•  Is  the  guidance  about  where  to  locate  observers  and  what  they  should  look  for  clear  and 
comprehensive? 

•  Are  the  data  collection  requirements  burdensome  to  the  observers?  Are  they  able  to  use 
the  information  in  their  AARs  with  the  units/personnel  being  trained? 

•  Could  the  format  of  the  assessment  packages  be  modified  to  improve  their  utility? 

•  Is  it  feasible  to  collect  the  information  specified?  For  example,  are  there  problems  with 
fielding  enough  observers  for  certain  types  of  exercises?  How  can  these  be  minimized? 

•  Are  the  data  collected  likely  to  be  invalidated  by  the  conditions  under  which  exercises 
are  conducted?  For  example,  if  the  maneuver  training  areas  are  too  small  for  doctrinal 
scenarios,  how  does  that  affect  the  validity  of  the  assessments?  Can  the  commander 
make  ‘mental  adjustments’  for  these  factors? 

•  Is  the  information  derived  by  using  the  assessment  packages  sufficient  to  allow  the 
commander  to  make  informed  decisions  about  the  course  of  his  training  program?  In 
particular,  is  there  sufficient  detail  on  staff  and  subordinate  unit  performance  to 
determine  how  to  remediate  their  weaknesses  prior  to  task  force  or  brigade  exercises? 

Field  trials  in  a  few  home  stations  would  provide  sufficient  information  to  conduct  a  cycle 
of  revisions  to  the  assessment  packages  to  make  them  more  useful  to  home  station  commanders. 
The  assessment  packages  should  be  tried  through  one  entire  training  cycle  culminating  in  a 
rotation  to  a  CTC.  This  would  enable  the  home  station  commander(s)  to  provide  information 
about  all  aspects  of  the  use  of  the  packages:  their  use  with  units  of  different  size  and 
composition,  the  ease  of  identifying  applicable  sections,  the  practicality  of  the  format,  the  utility 
of  the  data. 

D.  Field  Trials  at  Simulation  Centers 


Simulation  centers  (the  premier  example  of  such  centers  is  the  SIMNET  training  facility 
at  Ft.  Knox,  KY)  present  another  opportunity  for  valuable  feedback  about  the  utility  of  the  CCF 
Assessment  Packages.  One  of  the  advantages  of  simulations  is  the  repeatability  of  the  scenario 
and  conditions  of  the  training.  Using  the  assessment  packages  to  gather  information  about 
several  units  attempting  the  same  scenario  and  conditions  would  provide  the  opportunity  to  see 
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if  there  are  aspects  of  unit  performance  the  assessment  packages  do  not  capture  well. 
Furthermore,  the  scenario  and/or  conditions  may  be  varied  systematically  to  determine  how  they 
influence  the  information  gathered  using  the  assessment  packages.  In  naturalistic  settings  (e.g. 
home  stations  or  CTCs)  it  may  be  difficult  to  know  whether  the  shortcomings  of  the  assessment 
package  need  to  be  remediated  by  capturing  more  about  the  scenario  and  conditions  or  about  the 
performance  of  the  unit. 

Simulation  centers  at  home  stations  (in  some  cases  smaller  SIMNET  facilities  than  that 
at  Ft.  Knox;  in  other  cases  centers  that  run  computerized  battle  exercises  that  can  be  used  to  train 
commanders  and  staff  elements)  could  provide  similar  information  for  the  more  limited  exercises 
they  perform. 

Generally,  the  simulation  centers  have  sufficient  observers  to  provide  coverage  of  the 
actions  on  the  battlefield.  The  ability  to  capture  the  events  and  replay  them  also  may  permit  a 
smaller  corps  of  observers  to  capture  the  desired  information.  The  main  questions  to  be  answered 
by  feedback  from  usage  of  the  assessment  packages  at  simulation  centers  are: 

•  Can  the  commander  or  simulation  center  manager  easily  determine  which  components 
of  the  assessment  packages  are  useful  for  assessing  training  of  staff  elements  and 
subordinate  units,  and  tailor  the  assessment  accordingly? 

•  Is  the  guidance  about  where  to  locate  observers  and  what  they  should  look  for  clear, 
comprehensive,  and  adaptable  to  the  simulation? 

•  Could  the  format  of  the  assessment  packages  be  modified  to  improve  utility? 

•  Are  the  data  collection  requirements  burdensome  to  the  observers?  Are  they  able  to  use 
the  information  in  their  AARs  with  the  units/personnel  being  trained? 

•  Are  the  data  collected  likely  to  be  invalidated  by  the  deviations  from  realism  peculiar 
to  the  simulation?  Can  the  commander  or  OCs  make  ‘mental  adjustments’  for  these 
factors? 

•  Is  the  information  derived  by  using  the  assessment  packages  sufficient  to  allow  the 
commander  to  make  informed  decisions  about  the  course  of  his  training  program? 

Another  advantage  of  simulation  centers  is  that  they  can  run  a  full  schedule  of  training 
exercises  from  which  feedback  on  the  assessment  packages  can  be  derived.  In  home  station 
training  units  typically  do  not  perform  exercises  suited  for  use  of  the  assessment  packages  nearly 
as  frequently.  Thus,  feedback  data  should  accumulate  more  rapidly  from  the  simulation  centers. 

E.  Field  Trials  at  the  Combat  Training  Centers 

Like  the  simulation  centers,  the  CTCs  are  able  to  conduct  training  nearly  continuously. 
Data  on  the  application  of  the  CCF  Assessment  Packages  could  accumulate  quickly  from  the 
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CTCs.  These  centers  are  the  Army’s  premier  unit  training  facilitites.  They  are  rapidly  becoming 
significant  sources  of  doctrinal  review  and  testbeds  for  the  application  of  tactics,  techniques  and 
procedures  to  complement  doctrine.  Feedback  from  these  centers  would  provide  invaluable 
information  about  the  application  of  the  assessment  packages  to  current  practices  with  respect  to 
scenarios,  operations  and  assessment.  Furthermore,  the  CTCs  could  be  a  source  of  behavioral 
anchors  for  the  assessments  of  the  outcomes.  Or,  they  could  provide  a  substantive  review  of 
such  behavioral  anchors  developed  by  other  Army  agencies. 

The  fact  that  all  of  the  CTCs  except  the  Battle  Command  Training  Program  (BCTP)  are 
‘dirt’  CTCs  -  i.e.  the  exercises  are  conducted  using  real  equipment  on  doctrinally  appropriate 
terrain  --  means  that  the  physical  format  of  the  assessment  packages  may  require  adaptation  to 
field  environments.  The  OCs  at  the  CTCs  should  be  able  to  provide  considerable  insight  into 
ways  in  which  the  assessment  packages  could  be  modified  for  field  use^. 

The  questions  that  could  be  addressed  by  conducting  field  trials  at  these  centers  are 
essentially  identical  to  those  that  might  be  answered  by  feedback  from  simulation  center  field 
trials: 


•  Is  the  guidance  about  where  to  locate  observer/controllers  and  what  they  should  look 
for  clear,  comprehensive,  and  adaptable  to  the  CTC? 

•  Should  the  format  of  the  assessment  packages  be  modified,  especially  to  facilitate  their 
use  in  field  environments? 

•  Are  the  data  collection  requirements  burdensome  to  the  observer/controllers?  Are  they 
able  to  use  the  information  in  their  AARs  with  the  units/personnel  being  trained? 

•  Are  the  data  collected  likely  to  be  invalidated  by  the  deviations  from  realism  peculiar 
to  the  CTC?  Can  the  commander  or  OCs  make  ‘mental  adjustments’  for  these  factors? 

•  Is  the  information  derived  by  using  the  assessment  packages  sufficient  to  allow  the 
commander  to  make  informed  decisions  about  the  status  of  his  units  and  the  strengths 
and  wejiknesses  of  his  training  program? 

•  Is  the  information  derived  by  using  the  assessment  packages  suitable  for  developing 
long  term  trends  that  could  be  used  to  fulfill  the  CTCs’  second  mission:  providing 
feedback  to  the  Army  concerning  the  impact  of  changes  to  doctrine,  organization, 
training,  materiel,  leadership  and  soldiers? 


^  Electronic  data  gathering  instruments,  whose  utility  has  been  demonstrated  at  the 
National  Training  Center  and  the  Joint  Readiness  Training  Center  (Huffman,  1994;  Keesling, 
1994;  and  Root,  1994),  could  be  used  to  expedite  the  collection  and  forwarding  of  the  data  in 
all  settings. 
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IV.  COLLECTION  AND  ANALYSIS  OF  FEEDBACK  DATA 


The  proposed  review  by  the  school(s)  responsible  for  the  doctrinal  aspects  of  the  CCF 
could  be  conducted  in  paper  format.  Written  feedback  concerning  the  appropriateness  of  the 
assessment  packages  to  current  and  future  doctrine  would  be  most  useful. 

The  questions  outlined  in  each  of  the  above  sections  could  become  the  basis  for  a  data 
collection  instrument  that  would  be  used  to  gather  the  feedback  needed  for  a  round  of  revisions 
to  the  assessment  packages.  It  would  be  most  appropriate  to  pose  these  questions  in  interviews 
to  be  conducted  with  the  training  managers,  OCs,  and  unit  commanders  and  leaders. 

At  home  stations,  the  interviews  could  be  conducted  after  specific  training  exercises 
(CPX,  Company/Team  STX  and  FTX,  Task  Force  STX  and  FTX,  for  example).  At  the 
simulation  centers  and  the  CTCs,  the  interviews  "with  the  OCs  could  be  conducted  following  the 
AAR  for  a  mission.  Interviews  with  personnel  in  the  unit  being  trained  probably  should  be 
conducted  after  the  rotation  is  completed. 

Analysis  of  the  relationship  between  the  measures  of  effectiveness  and  the  assessments 
of  outcome  performance  would  be  of  particular  interest.  One  analysis  of  interest  would  be  to 
determine  whether  the  measures  of  effectiveness  and  assessments  of  performance  are  sensitive 
to  differences  among  units.  The  study.  Determinants  of  Effective  Unit  Performance  (Holz,  Hiller 
and  McFann,  1994:  particularly  Section  II)  found  that  there  were  very  large  differences  in  the 
training  programs  that  were  conducted  at  home  stations  and  that  these  were  reflected  in 
performance  assessments  conducted  at  the  National  Training  Center.  To  be  useful,  the  CCF 
Assessment  Packages  should  show  similar  sensitivity  to  unit  differences. 

A  second  question  of  interest  would  be  whether  the  performance  of  the  outcomes  is 
reliably  related  to  the  measures  of  effectiveness.  It  might  be  possible  to  establish  a  clear 
connection  between  patterns  of  outcome  attainment  and  quantitative  scores  on  measures  of 
effectiveness.  Such  relationships  could  help  to  focus  training  programs  by  indicating  which 
outcomes  must  be  achieved  to  attain  higher  measures  of  effectiveness.  Finally,  an  analysis  of 
the  relationship  of  home  station  assessments  and  measures  of  effectiveness  to  those  obtained  on 
the  same  units  at  the  CTCs  would  be  interesting  as  a  vehicle  for  providing  feedback  about 
performance  expectations  to  the  home  stations. 

IV.  SUMMARY 

The  CCF  Assessment  Packages  have  been  prepared  to  complement  a  function-oriented 
training  program  at  home  stations.  These  packages  would  also  be  useful  for  assessing  unit 
performance  in  simulated  combat  at  the  CTCs  and  other  simulation  centers.  Before  they  are 
adopted  widely  they  should  be  reviewed  for  doctrinal  content  (with  some  attention  paid  to  areas 
that  are  in  flux  and  likely  to  change  soon),  and  for  practicality  of  application  in  the  settings  for 
which  they  are  intended.  Field  trials  in  these  settings  should  provide  the  information  needed  to 
improve  the  utility  of  these  packages. 
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